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ABSTRACT 

The American public continues to be perplexed by the 
substandard reading, math, and science skills of students. Science 
information alone is growing at a rate that will make it very 
difficult for many, especially students with feeble science knowledge 
and skills, to become scientifically literate. This research 
describes the development, distribution, and evaluation of a 
biomedical oriented newsletter named BioRap for sixth to eighth grade 
students. BioRap is designed in a weekly reader type format and 
provides current, relevant information to children in a more 
cost-effective manner. Expert faculty at major univers i t i es , research 
scientists at drug companies and hospitals, veterinarians, 
physicians, etc. serve as content resources for BioRap. Based upon 
data gathered from teachers who have used BioRap, it was concluded 
that BioRap has a strong potential for use in the classroom. Its 
short length allows flexibility in when and how teachers can 
introduce BioRap into the curriculum, ^t is believed that BioRap 
shows great promise as a vehicle to improve the science literacy of 
adolescents . (ZWH) 
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The American public continues to be perplexed by substandard 
reading and math skills of students. There is now broad 
evidence that science accomplishment is equally weak. That 
is especially troubling, considering that scientific 
information doubles every 10 to 15 years (Wilson, 1993) . 

Students with feeble science knowledge and skills now face an 
increasing, cumulative deficit in their understanding and 
application of science. Biomedical and technological 
developments demand contemporary knowledge and language for 
informed consumption by the public. Textbook authors and 
publishers face major challenges maintaining currency in 
times of continual discovery. And schools are under great 
pressure to provide a strong science foundation to meet the 
demand of accelerating scientific sophistication. 

This research describes the development, distribution, and 
evaluation of a biomedically oriented newsletter, BioRap, for 
sixth to eighth grade students. BioRap is sponsored by the 
nonprofit Connecticut United for Research Excellence (CURE). 

CURE was established as a consortium of institutions in 
Connecticut that expressed concern over student science 
literacy. Initially, a speakers' bureau was created to offer 
informed professionals who would speak in classrooms free of 
charge. Teacher requests for CURE speakers became so great 
that another vehicle— BioRap— was developed to provide current, 
relevant information to children in a more cost-effective 
manner. BioRap was planned as a Weekly Reader format for 
biomedical topics of interest to school children. Its 
purposes include inexpensive, wide distribution, and coverage 
of newer information than is possible in texts. BioRap 
content is presented in a variety of nontraditional ways that 
distinguish it from the usual textbook orientation. The same 
experts— faculty at major universities in the state, research 
scientists at drug companies and hospitals, veterinarians, 
physicians, etc.— who were speakers for CURE served as content 
resources for BioRap. A professional writer was engaged to 
provide the actual content presentation. 

1 A paper presented at the annual meeting of the American 
Educational Research Association, New Orleans, April 1994. 
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Evaluation of BioRap took several directions. CURE had 
intended the newsletter would not just raise scientific 
literacy of students, but also be effective across a range 
of grades, ability, socioeconomic and gender groups. BioRap 
was additionally meant to be amenable to incorporation into 
various types of classroom use without losing effectiveness. 
The specific objectives of the evaluation were to measure 
cognitive gains relative to BioRap content, and student 
reactions to different presentation formats within each 
issue. Student and classroom characteristics (age, gender, 
ability grouping) , as well as method of use of BioRap (to 
award a grade, as supplementary material) were considered in 
relationship to student performance. 

After a year of successful use of BioRap within Connecticut, 
distribution of the newsletter was expanded to six states 
through additional financial support from Join Hands. The 
current multistate distribution includes schools in 
California, Massachusetts, New Jersey, North Carolina, and 
Pennsylvania. Issues of BioRap were made available, free of 
charge, to teachers for classroom use. Teachers in turn 
agreed to use BioRap with their students, in any way they 
chose. They were requested to provide evaluation data, 
collected on a single sheet, to CURE for analysis of the 
effectiveness of BioRap when distributed beyond the 
originally targeted state. 



Methods 

Each issue of BioRap was planned as an eight-page newsletter 
to be printed on heavy, recycled and biodegradable paper. 

The issues would be targeted at current health concerns in 
which research and development was progressing rapidly. Each 
issue would have particular "columns" of a specific format 
and covering current information relevant to the theme of the 
issue. These formats include: 

■a cartoon presentation of facts; 

■a description (with photos) of the application of the 
scientific method to a contemporary problem; 

■a focused profile of a health or science related career; 

■a crossword puzzle with clue definitions of words 
presented in the issue; 

■a section of bulleted information on disease prevention 
and treatment; 

■a "word jumble" with scrambled letters forming vocabulary 
words introduced in the issue; 

■a column with international extensions of issue 
information; and 
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■activities students can do on their own to explore content 
further . 

Issues thus far distributed in Connecticut include one on 
measles and rabies, and a second on Lyme disease. A third 
issue on feline leukemia was the first to be distributed 
across the six states. It was developed as a lead into an 
AIDS issue that is being used in classrooms now (April 1994) . 

An evaluation sheet specific to each issue was constructed 
for students to complete. For each BioRap issue, multiple 
choice or matching content-based items were developed. Items 
were pilot tested and analyzed using a small a group of 
representative students. Criteria for item retention were 
that items had difficulty indices between .25 and .75 and 
discrimination indices greater than .20. The items were 
assessed for content validity and representativeness across 
the entire issue. Ten items were included on the achievement 
test portion of each evaluation sheet. In addition to these 
core items, three items were included that covered content to 
be presented in subsequent issues of BioRap. These items 
were intended to gauge the exposure of students to upcoming 
topics and were considered "pretest" estimates. 

In addition to the content questions on the evaluation 
sheets, students indicated the letter grades they usually 
received in class, their age, and gender. They also awarded 
letter grades to specific sections of BioRap (A to their 
favorite and F to their least favorite) , and an overall grade 
(A through F) to the total issue. On a separate sheet, 
teachers indicated for each class whether it was ability 
grouped and if so, what ability it included; the grade level 
of the class; whether the BioRap content test items were 
treated as a graded activity or not; and if the questions 
were answered in an open- or closed-book situation. 

During the first year of distribution in Connecticut schools 
only, schools were identified as located in an urban, 
suburban or rural setting and by the socioeconomic status of 
the community the students came from. Once expanded 
distribution began, limited resources preempted 
identification of community socioeconomic status, so data 
were identified by state only. 

Initial Data Source and Analyses, within Connecticut, 
fourteen teachers in nine schools, each teaching multiple 
classes in grades six through eight, used BioRap during the 
first year of distribution (1992-1993). Schools were 
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originally selected by random sampling from a listing of 
schools in one major Connecticut county. This sample 
produced no school in a low income urban area. Subsequently 
an urban middle school was purposely included to increase 
diversity of school settings. 

Data from approximately 2,000 student responses to the first 
two issues of BioRap have been analyzed. Although other 
self-selected students, teachers, and schools used BioRap 
during the first year, their responses have not been included 
in the initial analyses. 

Initial Results. As a baseline estimate of science 
knowledge, means were calculated for the 3-item pretest 
portions that represented content on subsequent BioRap 
issues. For both issues of BioRap the means on these items 
showed approximately 40% accuracy in responding. By 
comparison, the overall average scores on the issue-specific, 
10-item achievement test showed about 74% correct responding. 
Correct answers declined over the grades from 85% (6th 
graders) to 67% (eighth graders) . Predictably, students in 
high ability classes outperformed those in middle ability and 
in low ability classes (79%, 73%, and 63% average 
performance, respectively) . There were no sex differences in 
performance. It did make a difference if the quiz was to be 
used as part of a grade: 83% correct. 

Economic disadvantage extracted no special toll on 
achievement. Low income urban students scored 66% correct as 
compared to 67% from their more advantaged peers. In the 
majority of cases, the BioRap achievement items were used as 
an ungraded assignment. Thus, these scores reflect student 
mastery without the incentive of a grade awarded. 

The mean grade assigned by students to BioRap, across both of 
the initial issues and across all grade levels, was a B. 
Interestingly, the simple correlation between BioRap grade 
awarded and student achievement based upon the 10-item tests 
was -.32, meaning that it tended to be rated higher by 
students who scored worse. Students also assigned A and F 
grades to favorite and "yuk" BioRap sections. Here, 
individual differences reigned: Many students liked each of 

the 12 sections graded, while others disliked the same ones. 
The cartoon piece, in both issues, received simultaneously 
the most As and the most Fs. Contrary to our predictions, 
the word sections (jumble and crossword) received many high 
marks from students. 
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Expanded Distribution Data and Analyses. Data from the third 
issue of BioRap— covering feline leukemia— have been collected 
and analyzed from the six state distribution. Nearly 4000 
students in grades six through eight completed the 10-item 
content test. Descriptive information about these students 
is given in Table 1. Achievement scores from these students 
shows remarkably consistent performance across BioRap issues 
regardless of content. Average score across the six states 
and three grade levels was 73.5% correct. Connecticut and 
Massachusetts students, in classes taught by teachers likely 
to have used BioRap during the previous year, achieved 
slightly higher average scores and showed smaller variability 
in scores than did students from other states (M = 75.5, 

78.1; SD = 19.4, 17.8 for CT and MA, respectively). 



Insert Table 1 here 



Student responses on the pretest items addressing AIDS 
content (information covered in the issue currently being 
distributed) showed higher science literacy than for 
previously covered topics. Average percent correct on the 
brief three-item pretest was 53.5, with California students 
showing notably higher scores (M = 60.6) and Pennsylvania 
students showing the lowest pretest performance (M = 47.4). 

The pattern of declining performance with increasing grade 
level that was noted initially was not evident in the 
expanded distribution data. Sixth and seventh grade students 
performed equivalently (72.8% and 72.9% correct, 
respectively) whereas eighth graders showed 75.5% correct 
responding. Student ability was again related to 
performance, with homogeneously grouped, high ability 
students achieving an average of 79.6% correct. 

Homogeneously grouped middle ability students showed 76.8% 
average performance, low ability and heterogeneously grouped 
classes 72.3% and 71.8% correct, respectively. There was 
again no practical difference in the performance of male as 
compared to female students (72.3% and 73.3% correct, 
respectively) . The condition of having the 10-item test 
count toward a classroom grade was related to achievement 
outcome: graded work produced 76.4% mastery, and ungraded, 

71.3% correct. An open-book testing condition showed 77.9% 
achievement, and closed book, 70.2%. A summary of the 
average performance of students, grouped by selected 
variables, is presented in Table 2. 
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Insert Table 2 here 



Overall grades awarded to BioRap by the student consumers 
were again squarely in the B range. Connecticut and 
Massachusetts youngsters liked it best (M = 1.97 and 1.99 on 
a scale of 1 = A to 5 = F) , with students from the other 
states showing average grades awarded between 2.10 and 2.30. 

As with previous issues, there was wide variation in student 
preferences for the sections of BioRap. All sections 
received a fair share of A and F grades, with the cartoon 
again receiving simultaneously the largest proportion of both 
As and Fs. And as before, word games (crossword and jumble) 
got high marks from many students. Some student respondents 
elected to award only As, because they could not identify any 
section deserving an F! No gender related differences were 
found in student preference for sections. A summary of the 
grades awarded on the six state distribution of the feline 
leukemia issue is given in Table 3 . 



Insert Table 3 here 



Data from the multistate distribution were subjected to a 
regression analysis to estimate what characteristics were 
most strongly related to student achievement on the 10-item 
achievement test. Two sets of variables were entered into 
the regression analysis in hierarchical order. The first 
set, arranged to represent demographic information, included 
data about student grade level (6th, 7th, or 8th, effect 
coded) , gender, state of residence (effect coded) , whether 
the testing condition was open or closed book, and if the 
test results would be counted toward a class grade. A second 
set of variables, constructed as proxy for student ability, 
included student self-report of typical classroom grades 
received and an effect coded indicator of the nature of the 
classroom grouping: heterogeneous; or homogeneously grouped 
as high, middle, or low ability. 

The set of demographic variables accounted for approximately 
8% of the variation in test scores (R = .29), with the 
open/ closed book and graded/ not graded test being the only 
significant predictors in the set. (Because the large sample 
resulted in excessive statistical power, significance tests 
were performed at a < .001.) Accompanying these variables, 
state of residence explained no additional variation in 
student test scores. Within the ability set of variables, 
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self-reported previous grades was the only significant 
predictor, increasing prediction to better than 14% (R = 
.38). Table 4 summarizes this hierarchical regression 
analysis . 



Insert Table 4 here 



Discussion 

The decline in achievement over grades 6 to 8 seen initially 
did not occur in the expanded distribution of BioRap. Aside 
from that difference, the expansion data generally follow the 
patterns noted initially. Lack of sex differences in 
achievement and grade awarded to BioRap runs contrary to the 
usual science stereotypes favoring males. Gender status does 
not appear relevant to achievement or satisfaction outcomes 
associated with BioRap. The comparison of pretest to 
posttest performance shows that students demonstrate 
significant and practically meaningful gains in knowledge of 
content after using BioRap. Gains occur under all sorts of 
teaching conditions, but achievement is maximized with the 
incentive of contingent grades for test results and when open 
book is allowed during testing. 

In a univariate view, between-state differences in student 
performance occurred. However, the multivariate 
perspective— using regression analysis— showed that in 
combination with other variables, between state differences 
evaporated. We take the regression results to be supportive 
of BioRap as a means to improve science literacy. Its 
effectiveness is independent of unalterable variables such as 
grade level, gender, and state of students' residence. 
Motivational variables (graded or not graded test results) 
and testing condition (open or closed book) are under teacher 
control and might be manipulated depending on teachers 
intentions about how students ought to use BioRap. In short, 
teachers have some latitude about instructional conditions 
with BioRap. 

The relationship between student ability, as represented by 
self-reported history of grades, was not unexpected. 

Students with higher achievement histories typically perform 
better in academic situations than do those with lesser 
previous achievement. The multiple choice, objective format 
of the posttest measure is a commonplace example of academic 
tasks students face in classrooms and it is consistent with 
factors that contribute to student grades. 
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Future assessment of outcomes of BioRap use might explore 
other less traditional outcome variables aside from academic 
achievement. One such alternate variable is student self- 
efficacy. Bandura (1986) has made the strong case that 
efficacy judgments have causal influence on behavior. Thus, 
if students' science self-efficacy is enhanced by BioRap, 
they are more likely to approach, engage in, and succeed at 
science-related tasks. If students show a stronger sense of 
confidence or efficacy for science skills or, ultimately, for 
careers in science, after BioRap exposure there would be 
additional support for the newsletter's classroom use. 
Assessment of self-efficacy as an outcome variable would also 
address interest in documenting effects of schooling beyond 
academic or achievement gains alone. 

Clearly students enjoy BioRap. The grades they award to what 
is essentially a classroom teaching tool are remarkably high 
and consistent across state, gender, and issue. There seems 
to be some section of BioRap that appeals to each student, 
and certain ones that appeal to many. Such stimulus variety 
improves attention and motivation (Owen, Froman, and Moscow, 
1981) , and judging from the spread of grades awarded to 
individual sections of BioRap, the diversity in newsletter 
format may improve student motivation for science 
achievement. The high grades given to the word puzzles are 
particularly encouraging. These sections give an additional 
reinforcement for learning beyond merely reading didactic 
presentations of information. They help students to rehearse 
information they were exposed to in some other part of each 
issue, as well as actively engaging students in practicing 
vocabulary. The enactive nature of the word games should 
help students to rehearse and place information into long 
term storage for future reference (Anderson, 1989) . 

BioRap has a strong potential for use in the classroom. Its 
slick newsletter formatting engages and interests students. 
Its short length allows flexibility in when and how teachers 
introduce BioRap into the curriculum. Production time for 
each issue has decreased: Only six months was needed to 

generate the most recent issue. The relatively inexpensive 
cost for production makes it amenable to frequent revision to 
maintain contemporary information with continued advances in 
the topics presented. Given evaluation data collected to 
date, BioRap shows great promise as a vehicle to improve 
science literacy of adolescents. 
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Table 1 



Descriptive Information for Sample* 





N 


% of Sample 


State 


California 


608 


15.5 


Connecticut 


480 


12.2 


Massachusetts 


1192 


30.3 


New Jersey 


601 


15.3 


North Carolina 


713 


18.1 


Pennsylvania 


336 


8.5 


Grade in school 


Sixth 


755 


20.6 


Seventh 


2225 


60.7 


Eighth 


681 


18.6 


Student Gender 


Male 


1530 


48.6 


Female 


1616 


51.3 


Test Results Counted 


for Classroom 


Grade 


No 


2064 


55.2 


Yes 


1673 


44.8 


Testing Condition 


Open book 


1209 


39.2 


Closed book 


1259 


40.8 


Classroom Grouping 


Heterogeneous 


2587 


70.6 


High ability 


498 


13.6 


Middle ability 


249 


6.8 


Low ability 


255 


7.0 



* Total sample=3932 cases. Some percents total less than 100 



due to missing data. 
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Table 2 

Average Test Performance by Selected Grouping Variables 



Mean 


% Correct 


SD 


State 


California 


69.96 


22.15 


Connecticut 


75.54 


19.45 


Massachusetts 


78.08 


17.78 


New Jersey 


72.69 


20.57 


North Carolina 


68.68 


20.49 


Pennsylvania 


68.98 


20.13 


Grade 


Sixth 


72.81 


20.06 


Seventh 


72.92 


20.33 


Eighth 


75.47 


19.41 


Student Gender 


Male 


72.30 


20.60 


Female 


73.30 


19.61 



Test Results Counted for Classroom Grade 



No 


71.34 


20.60 


Yes 


76.34 


19.09 


Testing Condition 


Open book 


77.91 


19.25 


Closed book 


70.17 


19.80 


Classroom Grouping 


Heterogeneous 


71.76 


20.63 


High ability. 


79.63 


17.12 


Middle ability 


76.77 


17.49 


Low ability 


72.28 


21.08 
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Table 3 



A and F Grades 


Awarded to 


Sections of BioRap 


% of 


As Awarded 


% of Fs Awarded 


Cover story 


13.9 


10.8 


Today ' s Research Lab 


6.8 


9.7 


Career Rap 


3.4 


14.9 


In Fact Bullet Bits 


2.4 


5.6 


Crossword 


12.1 


10.4 


Bag It Activities 


2.8 


8.6 


Cartoon 


23.8 


11.9 


Body System 


5.1 


7.3 


Rap Sheet Fast Facts 


3.0 


6.5 


Word Jumble 


17.1 


4.6 


Facts Around the World 


4.1 


6.7 


Photographs 


5.4 


2.8 



Grades awarded from multistate distribution only. Percents 
do not total to 100 because of missing data. 




13 



Science Literacy 
Page 13 



Table 4 

Summary of Hierarchical Regression 



Variable 


b 


se b 


I : 3 


F 


P 


Demographic Set 








14.13 


<.001 


sex 


.55 


.93 


.01 






graded/not graded 


3.61 


1.15 


.09 




<.001 


open book/not 


-5.99 


1.27 


-.16 




<.001 


CA 


-2.71 


2.12 


-.06 






CT 


2.47 


2.49 


.05 






MA 


5.32 


2.06 


.11 






NJ 


3.21 


2.43 


.06 






Nc 


-6.81 


2.45 


-.14 






grade 6 


.26 


1.65 


.01 






grade 7 

Ability Set 


-3.56 


1.77 


-.09 


27.24 


<.001 


GPA 


3.19 


.34 


.23 




<.001 


group : hetero . 


1.73 


1.64 


.04 






group: high abil. 


7.17 


2.87 


.09 






group: middle 


5.56 


2.44 


.07 
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